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A VIRTUAL DEVICE 

Technical field 

5 This invention relates to mobile coniniunication systems and 
mobile data services in general and in particular a Virtual 
Device for use in such systems. 

10 Technical background 

Although mobile and portable devices are getting more and 
more advanced and incorporating more functions, it is 
foreseen that in the future a user will make use of more 

15 and more devices such as mobile phone, PDA (Personal 

Digital Assistant), headset, microphone, digital camera, 
digital camcorder, etc. The user will have to manage all of 
them and it is not always an easy task even for 
technicians. It would be nice if the user can consider all 

20 the independent devices as one big "Virtual Device" having 
multiple input and output units and providing a coherent 
and surround interface to the user [1], [2]. 

Commonly a number of add-on units are available for the 
users of mobile devices (cellular telephones and PDAs), 

25 like head-sets, web cameras, external keyboards, printers, 
etc. These units can be connected to the mobile device 
through e.g. a cable, an infrared link or a Bluetooth® 
link. However, the mobile device must be designed for 
handling these particular external units, and comprise the 

30 necessary interfaces and software solutions. 

An expansion of this concept is disclosed in Norwegian 
patent application no. 1998 5563 to Aktiebolaget LM 
Ericsson, now abandoned. This application describes a 
component-based terminal, in which the main component (a 
35 processing unit for a mobile telephone) is carried by the 

user, and in which secondary components; head-sets, display 
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units, etc., either can be carried on the users body, or be 
available in the near environment. Thus, the terminal is 
broken up into many separate functional elements. This 
means that the resultant teinninal can include external 
5 facilities like a large stationary display screen, a full 
size keyboard or a printer that are available in the room 
where the user is situated at the moment (e.g. in his 
office) ; the terminal being configured on-the-fly in order 
to add in any components that are available. The 

10 configuration is controlled by software agents for each 

individual external unit; the agents being contained in the 
main component. However, the application does not furnish 
any more information on how the components interact or how 
the terminal is configured. The setup is locked to the 

15 features which are supported by hardware and software' in 
the main component. 

From F. Louagie, L. Mufioz, S. Kyriazakos, "'Paving the Way 
for the Fourth Generation: A New Family og Wireless 
Personal Area Networks", IEEE 56^^ Vehicular Technology 

20 Conference, Vancouver, Canada, September 2002, there is 

known a wireless personal network comprising a master unit 
controlling a number of primitive add-on units. This is an 
elaborate version of the personal network; in principle 
known from mobile phones with add-on units like microphones 

25 and headphones. 

European Patent Application EP 1282282 A2 describes a 
personal network in which the JINX service is used to 
detect applications installed on the units forming the 
network. However, the JINX service cannot discover and add 
30 hardware components to a network; it is only able to 
recognize applications or services. 

Brief siaznmaxy of the xnven-bxon 

35 It is an object of the present invention to provide a 

Virtual Device that combine the abilities and resources of 
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several independent devices possessed by the user^ like 
mobile phones, PDAs, etc., by which the user does not have 
to purchase the resource twice and hence can save money. 

Another object of the invention is to provide a Virtual 
5 Device in which the individual devices share resources like 
processing power, battery power, memory, data, 
applications, network access points and user-interfaces. 

Still another object is to provide a Virtual Device in 
which the user stores his setup preferences in a user 
10 profile avoiding having to enter his preferences in- each 
individual device . 

The objects above are solved in a Virtual Device according 
to the present invention as claimed in the appended patent 
claims . 

In particular, according to a first aspect, the invention 
relates to a Virtual Device composed of a multitude of 
stationary and/or portable electronic devices, of which at 
least one of the devices is a first Open Device with an 
available inner structure and a network interface, at least 
one Primitive Device arranged to communicate with other 
Devices and act as a pure slave to said devices, 
said Virtual Device further including at least one Closed 
Device with a closed inner structure and a network 
interface, , said open and closed devices being connected 
in a Personal Area Network, a Personal Area Network 
middleware being installed in said first Open Device, said 
first Open Device being arranged to act as a Personal Area 
Network controller. 

According to a second aspect, the invention relates to a 
30 Virtual Device composed of a multitude of stationary and/or 
portable electronic devices, including an Open Devices with 
an available inner structure and a network interface, at 
least one Primitive Device arranged to communicate with 
other devices and act as a pure slave to said devices. 
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which is characterized in at least one additional Open 
Device, at least one Closed Device with a closed inner 
structure and a network interface, said open and closed 
devices being connected in a Personal Area Network, a 
5 Personal Area Network middleware being installed on said 
Open Devices, said Personal Area Network Middleware being 
distributed between said Open Devices, said Open Devices 
being arranged to act as a Personal Area Network 
controller. 

According to a third aspect, the invention relates to a 
Virtual Device composed of a multitude of portable 
electronic devices, including an Open Device with an 
available inner structure and a network interface, at least 
one Primitive Device arranged to communicate with other 
devices and act as a pure slave to said devices, 
which is characterized in at least one Closed Device with a 
closed inner structure and a network interface, said at 
least one Closed Device being equipped with a network 
interface connected to an external network which includes a 
Personal Area Network server, a Personal Area Network 
middleware being installed on said server, said open and 
closed devices and said server forming a Personal Area 
Network operated by said Personal Area Network middleware. 

According to a fourth aspect, the invention relates to a 
25 Virtual Device composed of a multitude of stationary and/or 
portable electronic devices, which includes a Closed Device 
with a closed inner structure and a network interface 
connected to an external network, at least one Primitive 
Device arranged to communicate with said Closed Device or 
30 devices and act as a pure slave to said device or devices, 
a server in said external network on which is installed a 
system and network management software, said server being 
arranged to communicate with said Closed Device or devices 
in order to act as a Personal Area Network controller. 
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Brief description of the drawings 

The invention will now be described in reference to the 
5 appended drawings, in which 

Figure 1 is an overview showing devices that might be 
included in a Virtual Device in a Personal Area Network 
according to the present invention. 

Figure 2 is a schematic diagram showing the overall 
10 architecture of a Personal Area Network, 

Figure 3 is a diagram showing a Virtual Device on a PAN 
with a unique Open Device, 

Figure 4 is a diagram showing a PAN application structure. 

Figure 5 is a diagram showing the organization of the 
15 Personal Area Network Middleware for a Virtual Device on a 
PAN with a unique Open Device, 

Figure 6 is showing a Virtual Device on a PAN with multiple 
Open Devices, 

Figure 7 is showing the Personal Area Network Middleware 
20 for a Virtual Device on a PAN with multiple Open Devices, 

Figure 8 is showing a networked PAN without any Open 
Device, 

Figure 9 shows a Virtual Device on a networked PAN with an 
Open Device, 

25 Figure 10 shows the Personal Area Network Middleware for a 
Virtual Device on a networked PAN with an Open Device. 

Detailed descuriptxon of the invention 

With the emergence of local short-range communication 
30 technologies such as Bluetooth [3], WLAN [4], HiperLAN, 
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etc. local connectivity between devices is enabled. The 
devices belonging to a person will form a private Personal 
Area Network (PAN) where devices appear and disappear 
dynamically. However, most importantly, connectivity does 
5 not necessarily mean communication and many challenging 

issues must be resolved before devices can collaborate and 
together form a Virtual Device. This invention proposes and 
describes a PAN middleware that enables the formation of a 
Virtual Device on the PAN and that is capable of handling 
10 the dynamical presence of devices and the diversity of 
device types. 

This invention can be realized in at least four different 
embodiments : 

1- An architecture and the necessary middleware functions 
15 to realize a Virtual Device based on an isolated PAN 

(Personal Area Network) with a unique open device 

2. An architecture and the necessary middleware functions 
. to realize a Virtual Device based on an isolated PAN 
with multiple open devices. 

20 3. An architecture and the necessary middleware functions 

to realize a Virtual Device based on networked without 
open device 

4. An architecture and the necessary middleware functions 
to realize a Virtual Device based on networked with 
25 open device 

In order to realize the Virtual Device, the PAN and his 
devices must be hidden to the user and his applications and 
this is done by introducing a PAN middleware. 

30 As shown in Figure 2 the PAN middleware must be capable of 
handling not only the Personal Area Network but also the 
heterogeneity and the dynamic of devices. There are 
precisely these two characteristics that make the PAN 
middleware different from the traditional Distributed 
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Computing Middleware where the computing nodes are supposed 
to be stable and equal or at least having the minimum of 
capabilities e.g. processing, storage, communication, etc. 

5 Indeed, on the PAN there are a variety of devices such as: 

• Communications devices (fixed, mobile &IP phones) 

• Computing devices (Computer, PC, laptop, PDA, etc.) 

• Peripheral devices (Printer, scanner, digital camera, 
camcorder, etc.) 

10 • Electronic devices (TV, stereo equipment, 

loudspeakers , etc . ) 

• Electronic appliances (fridge, espresso machine, 
washing machine, etc.) 

The devices can be: 

• On-body or Off Body 

• Mobile or stationary 

• With or without Network connections 

• With or without local wireless connections 

It will be necessary to differentiate the device types as 
follows : 

• Primitive device: Simple devices that cannot operate 
alone but are slave to other devices, e.g. earphone, 
microphone, display, etc. 

• Open device: Devices allowing the installation and 
execution of PAN Middleware and other applications, 
e.g. PDA, laptop, workstation, etc. 

• Closed device: Devices that can operate stand-alone 
but are like a black box i.e. its internal structure 
and functions are totally hidden and communicate with 
the environment though only well defined interfaces. 
For instance, a digital camera is advanced device 
having processor, memory, input and output units but 
does not allow the installation and execution of 
foreign applications. It communicates with the rest of 
the word via interfaces like serial, parallel, USB, 
etc. 
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Let us now successively consider the following four 
solutions as follows: 

• Virtual Device on isolated PAN with unique Open device 
5 • Virtual Device on Isolated PAN with multiple Open 

devices 

• Virtual Device on Networked PAN without Open device 

• Virtual Device on Networked PAN with Open device 

VIRTUAL DEVICE ON ISOU^TED PAN WITH UNIQIOE OPEN DEVICE 

10 

In this PAN configuration, there is no device having 
connection with any wired or wireless network. There is 
only one Open device while the rest is either Primitive or 
Closed. Figure 3 shows a Virtual Device on a PAN with two 
15 primitive devices: microphone and earphone, two Closed 

devices: Mobile phone and Camcorder, and one Open device: 
PDA. 

In such a PAN it is desirable to achieve the following: 
20 • All the Open and Closed devices should have the 

ability to use the Primitive ones i.e. the mobile 
phone, the camcorder and the PDA should be able to 
share the microphone and the earphone. It is worth 
noting that today, such a device sharing is not 
25 supported with Bluetooth. 

• The Open devices should have the ability to use Closed 
devices. This is possible although very dependent on 
the interfaces offered by the Closed device. For 
example, the PDA can only communicate with the mobile 

30 phone using the AT command [5] to make e.g. a call 

request . 

• The Closed devices can only user other Open or Closed 
devices if they are programmed for that. 



35 A PAN middleware is necessary to achieve the resource 
sharing as above. In this case the PAN middleware is 
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located on the PDA since it is the unique device capable of 
accommodating the PAN Middleware (PANM) . 

Let us now define the necessary capabilities and functions 
5 for this configuration. 

A. Device Profile 

The PANM should be able to detect and to recognize what are 
10 the devices present in the PAN, what type they are and what 
capabilities they have or what service they are offering, 
e.g. printing, sound input, text displaying, etc. The PANM 
must have the mechanism to handle the device profiles 
containing device type and capabilities and to do the 
15 comparison with detected devices. The definition of device 
types must be logical and unambiguous definition of device 
types. It must be logical to address both composite devices 
and inherited devices . It must be unambiguous to guarantee 
the identification of a device type. Ideally, the 
20 definition of device types should be standardized. 

B. Resource Discovery and Updating 

The Resource Discovery comprises both the Device and 
25 Service Discovery. To support the PAN's dynamic where 

devices appear and disappear, the PANM should be equipped 
with mechanisms for Device Discovery and Updating. One 
alternative is to carry out polling periodically. Since 
Primitive devices may be tied to their master and not 
30 visible to other, the PANM should also ask also Open and 
Closed devices for Primitive devices connected to the 
device. In a second alternative, the PANM only run the 
device discovery once and relies on triggering from the 
network layer to update its device set. In a third 
35 alternative, the changes are only updated when an attempt 
to reach a device upon request from an application fails. 
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The PANM should also store the present devices. It is hence 
necessary to have a naming convention, e.g. PDA 1, Mobile 
2, etc. The network addresses (IP, Blutetooth, etc.) and 
protocols for each device must also be saved. 

5 

C. Device Xnpu-b and Output redirection 

The PANM should be able to redirect an output stream from 
one device to the input stream of another one. For example 

10 the sound output stream from the PDA can be redirected to 
the earphone and the sound input redirected to the 
microphone. It must intercept the output stream addressed 
to a device at an output port, perform the necessary 
transformation and send it to another device on the PAN. An 

15 API (Application Programming Interface) should be defined 

D. Application/ Service Input and output redirection 

The input and output redirection should also be done at the 
20 application/service level, i.e. one application can be 
redirected to one device while a second one to another 
device. However, this capability is more difficult since it 
depends on the feature of the operating system and may also 
require a new application structure as shown in Figure 4. 
25 An application consists of three components: Core, Input 
and Output. The Core component contains the main logic of 
the application and should remain invariant. The Input and 
Output Components should be specified according to the 
services supported such as sound input, text input, graphic 
30 display, colour printing, etc. If a device offers a service 
matching the one required then it can be used for the 
application. 

E. User Profile Management 

35 

The user and owner of the PAN should be reserved the right 
to define the following: 
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• What devices are allowed to participate to his PAN 
when present within the PAN's coverage. 

• How the devices should be used at a particular 
location or at a particular time or according to a 
certain timetable. How should the redirection between 
devices be 

• Which device should be internal and which one should 
be global and visible to the outside world. 

• Which applications and their settings 

• How should the applications should be distributed 
among the devices, i.e. how the three components 
should distributed among devices according to criteria 
as quality, price, battery level, etc. 

In addition, the user must have the ability to alter, 
remove, and add device and applications at any time and 
anywhere. These hard requirements put quite a lot of 
challenge on the design of the User Profile structure and 
the architecture of the system around it. 

F. In-berface to the user 

Since the user, as owner of the PAN, should be the one 
having the right to decide everything, there is a need for 
an application called PAN Control that offers interfaces 
allowing him/her to communicate and control the PAN 
Middleware. As other applications, the PAN Control should 
be also structured as Core, Input and Output and should 
support a variety of input and output services. 

6. Interface to appllca1:±ons and services 

With the apparition of the PAN, a brand new type of 
applications called PAN-based applications is born. This 
new type of applications will actively take advantage of 
the unique properties of the PAN: device dynamic and 
resource sharing. A typical example of PAN-based 
applications is an application, which focuses on the 
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control of existing applications, their parallel execution, 
their flexible and dynamic composition, the distribution of 
their input and output. The PANM should provide an 
Application Programming Interface (API) allowing the 
5 application access to the PAN capabilities and functions. 
Such an API can be implemented in different technologies, 
e.g. Java, CORBA, XML Web service, etc. 

H . Summary 

10 

Generally^ to realize a Virtual Device on an isolated PAN 
with a unique Open device the PAN middleware must be 
installed and executed on this unique Open device. 

As shown in Figure 5 the necessary capabilities and 
functions in the PAN Middleware (PANM) in the case of an 
isolated PAN with unique Open Device are: 

• Device Profile 

• Resource Discovery 

• Device Redirection 

• Application Redirection 

• User Profile Management 

• PAN Control 

• PAN API 

This PAN configuration depends totally on the central 
device, the open device and will collapse when the open 
device disappears, is switched off or runs out of battery. 
Another disadvantage is the limitation of the resource 
sharing since most of devices are Closed and Primitive and 
hence impenetrable. 

VIRTUAL DEVICE ON ISOLATED PAN WITH MULTIPLE OPEN DEVICES 

In this PAN configuration, there aire now multiple Open 
35 devices, multiple Closed devices and multiple Primitive 

devices. Figure 6 shows a Virtual Device on a PAN with two 
Primitive devices: microphone and earphone, two Closed 
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devices: Mobile phone and Camcorder, and three Open 
devices: PDA, Laptop and PC. 

In such a PAN, in addition to the features identified for 
5 the previous case it is desirable to achieve better 

collaboration and more optimal resource sharing between the 
Open devices. The PANM as defined in the previous case can 
now be distributed among the Open devices - 

10 We present now successively all the possible alternatives 
for the distribution of the PANM (PAN Middleware) and 
resource sharing on the Open devices. 

A. Dxs-bributed Opera-blng System 

15 

In this alternative, the PAN Middleware is a Distributed 
Operating System deployed on all the Open devices and 
coupling them tightly as one. Such an alternative demands 
the implementation of the Distributed OS on all the Open 
20 device types and with the rapid growth in type of mobile 
devices this could be difficult. Another disadvantage is 
that the device dynamic can create serious problem for the 
Distributed OS that requires much stability. 

25 B. Dls-brlbuted Computing 

In this alternative, the PAN Middleware is a Distributed 
Computing Middleware, which hides the distribution and 
communications from the applications and application 

30 developers- In order to be able to support PAN, the 
traditional Distributed Computing Middleware must be 
extended with functionality to cope with the dynamic of 
devices. This alternative has an advantage compared to the 
previous in the sense that there are fewer operating 

35 systems and the number of Middleware implementations is 
lower. On the other, it is important to quantify the 
overhead introduced and to investigate whether it is 
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acceptable for mobile devices with limited processing and 
storage capabilities and battery life. 

C. XML Web Services 

5 

In this alternative, the PAN Middleware consists of the 
logic necessary to expose the capabilities and functions of 
each Open device as an XML (extensible Markup Language) Web 
services [6] . The other Open devices can hence invoke these 

10 Web services by using SOAP (Simple Object Access Protocol) 
[7] . With the Web service concept unlike the Distributed 
Computing the distribution is not hidden. Entities know 
that communicate with remote entities but communication is 
made simpler by the automatic generation of SOAP message, 

15 This alternative requires that a Web service platform is 

installed on each Open device offering Web service and this 
could create an overhead to the PAN. However^ the devices 
are more loosely coupled in this alternative and it can 
cope better with device dynamic. 

20 

D. Appllca^xon dis1:riJbu'blon, coord±nat:±on amd con-brol 

In this alternative the PAN Middleware performs the 
distribution, coordination and control of applications 

25 based on the application structure: Core, Input and Output. 
The distribution of the input and output components is 
already taken care by the Application Redirection Function 
mentioned earlier. Concerning the Core, it could only be 
moved and resumed on compatible Open device, i.e. devices 

30 capable of supporting it. For example, an application Core 
written in Java can only be moved to a device having a Java 
Virtual Machine and not to a MS Windows device. To achieve 
the application migration, the mobile agent concept can be 
appropriate since an agent can stop the execution of an 

35 application, serialize it, transport it to another device, 
de-serialize and resume the execution. 
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Generally^ to realize a Virtual Device on an isolated PAN 
with a multiple Open devices the PAN middleware should be 
5 distributed among the Open devices. To achieve the 

distribution of the PANM and resource sharing between Open 
devices a new function is introduced in the PANM called 
Resource Sharing. The Resource Sharing can be realized by 
one or combination of the approaches: Distributed Operating 

10 System r Distributed Computing^ XML Web Services or 
Application distribution^ coordination and control. 
To achieve optimal resource usage a Resource Sharing 
function is required in the PANM as shown in Figure 7. The 
necessary capabilities and functions in the PAN Middleware 

15 (PANM) in the case of an* isolated PAN with multiple Open 
Devices are: 

• Device Profile 

• Resource Discovery 

• Device Redirection 

20 • Application Redirection 

• User Profile Management 

• Resource Sharing 

• PAN Control 

• PAN API 

25 VIRTUAL DEVICE ON NETWORKED PAN WITHOUT OPEN DEVICE 



In this configuration the Virtual Device does not have any 
Cpen device but only either Primitive or Closed device. 
Usually as mentioned earlier, without an Open device,, the 
30 Virtual Device will collapse but fortunately there is a way 
to remedy the situation if there is one device with network 
connection - 



As shown in Figure 8, the mobile phone allows communication 
35 with a Computer running PAN Middleware. In this case the 
PAN can function very well because it is logically 
equivalent to the configuration Isolated PAN with unique 
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Open device if the Computer on the network is considered as 
a PAN device. This configuration is, however, very 
interesting since it opens for several relevant usage. 
Indeed, the network computer can be Home PC or the office 
5 workstation that the user always leave behind when 

traveling but it can still participate and give support to 
his PAN. This network computer can also be provisioned and 
managed by a Service Provider that offers PAN service to 
the users. 

10 

Generally r it is possible to realize a Virtual Device on a 
networked PAN without any Open Device if the user has an 
15 Open device on the network (Internet) running the PANM. 

1.1.1. VIRTUAI. DEVICE ON NETWORK P2^ WITH OPEN DEVICE 

In this configuration, the PAN has all the device types: 
Open, Closed and Primitive but also those having network 
connection. This configuration is logically equivalent with 

20 the Isolated PAN with multiple Open devices- However, as 
shown in Figure 9 in order to reduce the usage of the 
network connection, it is perhaps desirable not to have 
tight coupling between the PDA on the PAN and the network 
computer. The network computer in this case acts as a safe 

25 storage and a Synchronization function should be added in 
the PANM (PAN Middleware) as shown in figure 10. The 
synchronization function ensures the consistency of the 
Virtual Device data, settings and profiles. 

30 It is also quite useful to include a Device Software 
Installation and Updating function that offer the 
possibility to search, select, download and install the 
appropriate software version for a device on the PAN. When 
the user wants to add a new device in his Virtual Device 

35 specific software, e.g. drivers, plug-ins, etc. may be 
required. The Device Software Installation and Updating 
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function may carry out the necessary operation and allow 
seamless introduction of new devices to the Virtual Device. 

SUMMARY 

5 In a Virtual Device on a networked PAN with Open Device ^ 
the Open Devices both on the PAN and on the network 
(Internet) should run the PANM, 

The necessary capabilities and functions in the PAN 
10 Middleware (PANM) in the case of a networked PAN with Open 
Device are: 

• Device Profile 

• Resource Discovery 

• Device Redirection 

15 • Application Redirection 

• User Profile Management 

• Resource Sharing 

• Synchronization 

• Device Software Installation 
20 • PAN Control 

• PAN API 
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